1. Field of the Invention
The present invention relates to a technique of controlling series of image processing carried out for image data by multiple devices connecting with a communication path.
2. Description of the Related Art
The widely known technique utilizes a network like the Internet to download image data and output images from an image output device. Typical applications of the output images are displayed images on an image display device and printed images with a printer. Cellular phones that are capable of gaining access to the Internet may be used as the image display device.
Digital image data are often subjected to diverse series of image processing in the course of image output. The diverse image processing includes, for example, a halftoning process that reduces the number of tones expressed in each pixel, a color conversion process that converts the color system of master image data to the color system of an image output device, and a correction process that corrects the contrast or the tone of image data. Execution of the image processing according to the output characteristics of the device effectively enhances the picture quality of output images.
The device having a relatively low processing capacity, such as the cellular phone, requires an undesirably long processing time for image processing and image display. One possible countermeasure simplifies the image processing to shorten the processing time. But this method does not sufficiently enhance the picture quality of displayed images.
A server may alternatively carry out a required series of image processing. But this method does not improve the picture quality of image data, which is transmitted from a server without the image processing capacity. There are a diversity of image output devices and their applications. The uniform image processing executed in the server may thus not sufficiently enhance the picture quality.
These problems are not restricted to the cellular phones used as the image receiver device, but are common to any image receiver devices that have relatively low processing capacities and are used to output images. Similar problems to those arising in the course of image output are found in the system where the server receives image data transmitted from an image input device, such as a digital camera, via the network.
The object of the present invention is thus to solve these problems and to provide a technique that carries out required series of image processing according to characteristics of an image transmitter device and an image receiver device in the course of inputting or outputting images via a network or another communication path, so as to attain high-speed processing and sufficiently enhance the picture quality of resulting images.
In order to attain at least part of the above and the other related objects, the present invention combines the functions of an image transmitter device that transmits image data via a communication path with the functions of an image receiver device that makes input image data undergo a required series of image processing, thus attaining the high-speed processing and sufficiently improving the picture quality of resulting images. Namely the technique of the present invention selectively uses the image processing functions of the image transmitter device and the image receiver device.
In the system that downloads image data from a server connecting with the communication path and outputs images from an image display device or a printing device, the server functions as the image transmitter device, whereas the image display device or the printing device functions as the image receiver device. In the system that uploads image data read with a digital camera or a scanner to a server, the digital camera or the scanner functions as the image transmitter device, whereas the server functions as the image receiver device. The server may individually constitute the image transmitter device or the image receiver device, or a plurality of servers may be combined together to take charge of the processing in a distributed manner.
The selective use of the image processing functions of the image transmitter device and the image receiver device ensures the improvement in picture quality to some extent, excluding the special case in which neither of the image transmitter device and the image receiver device has the image processing functions. For example, the image receiver device enables image data transmitted from an image transmitter device without the image processing capacity to be subjected to a required series of image processing. With regard to a certain series of image processing executable by both the image transmitter device and the image receiver device, the technique of the present invention selects either the image transmitter device or the image receiver device for execution of the image processing by taking into account the image processing capacities of the image transmitter device and the image receiver device and the details of the image processing. This arrangement attains the high-speed processing and further enhances the picture quality of resulting images. The construction of the present invention to actualize the selective use of the image processing functions is discussed below in the order of the image transmitter device and the image receiver device.
The image transmitter device of the present invention includes an image processing unit, a control signal generator, and a transmitter unit. The image processing unit causes input master image data to be subjected to a specified series of image processing. The details of the image processing are set according to the type of the image receiver device. The master image data may externally be input into the image transmitter device or may internally be stored in the image transmitter device.
The control signal generator generates a control signal for controlling details of image processing to be executed by the image receiver device, based on the specified series of image processing carried out by the image processing unit of the image transmitter device. The control signal includes information used to identify the series of image processing executed by the image processing unit, if any. The control signal may follow an adequately selected format.
The transmitter unit transmits the control signal and the processed data to the image receiver device. The control signal may be attached to the processed data for simultaneous transmission or may be separately transmitted like transmitting the processed data after transmission of the control signal. As discussed later, the image receiver device changes the details of the image processing executed therein in response to the control signal. This arrangement thus ensures selective use of the image processing functions of the image transmitter device and the image receiver device.
When the image transmitter device has the higher image processing capacity than that of the image receiver device, the image transmitter device carries out the series of image processing, which imposes a significant load on the image receiver device, and the image receiver device is thus released from the image processing. This arrangement ensures execution of the image processing by the device having the higher processing capacity and thus shortens the processing time required for the image output. The image transmitter device may carry out the series of image processing, which is equivalent to the processing executable by the image receiver device or which is more complicated to ensure the further improvement in picture quality than the processing executable by the image receiver device. The image receiver device may take charge of the image processing, according to the image processing capacities of the image transmitter device and the image receiver device.
The image data transmitted from the image transmitter device without the image processing function does not include the control signal. The image receiver device thus readily specifies no execution of any image processing. In such cases, the image receiver device carries out a required series of image processing to enhance the picture quality.
It is desirable that the image data and the control signal of the present invention follow a common format, which is used for image data transmitted from the image transmitter device without the image processing capacity, or have upper compatibility. In one preferable application, a vacant bit included in a header of the image data transmitted without the image processing may effectively be applied for the control signal. In the case of transmitting image data in a specific language, such as HTML (Hyper Text Markup Language), that uses a tag to control the operations of the image receiver device to some extent, the control signal may be defined with the tag. Transmission of the control signal according to the common format used for the conventional image data enables the image receiver device to readily distinguish the image data transmitted from the image transmitter device of the present invention from the image data transmitted from another image transmitter device. The technique of the present invention is, however, not restricted to such applications, but may adopt a unique format or communication protocol proper to the image transmitter device or the image receiver device of the present invention.
The technique of the present invention may be applied to a system, in which a server is used as the image transmitter device and an image output device that outputs images is used as the image receiver device. In one application of this system, when the image receiver device is an image display device having a smaller number of expressible tones than a number of tones included in master image data, the image transmitter device may carry out a dispersed-type color reduction process as the specified series of image processing.
The color reduction process reduces the number of tones expressed in each pixel of the master image data to the number of tones displayable by the image display device. The dispersed-type processing prevents local concentration of pixels having an identical tone values in the processed image data after the color reduction. The dither method and the error diffusion method are known examples of such color reduction.
A known method of color reduction divides a master image into a plurality of divisions in the ascending order of tone values, and successively allocates the tone values after the color reduction to the respective divisions. This method is hereinafter referred to as simple color reduction. The simple color reduction unequivocally maps the tone value of the master image to the tone value after the color reduction. The dispersed-type color reduction, on the other hand, does not unequivocally map the tone value of the master image to the tone value after the color reduction and is thus more complicated than the simple color reduction. The dispersed-type color reduction, however, effectively prevents the occurrence of a quasi-contour after the color reduction and advantageously ensures the better picture quality. Application of the dispersed-type color reduction to the image transmitter device desirably shortens the processing time and enhances the picture quality. The arrangement of using the control signal to control the image processing executed by the image receiver device effectively prevents deterioration of the picture quality, due to the duplicated execution of the color reduction process.
In another application, when the image receiver device is an image display device having a non-linearity of expressible tones, the image transmitter device may carry out a tone correction process to compensate for the non-linearity as the specified series of image processing. The non-linearity here means a non-linear relationship between the tone value and the actually displayed brightness. There is accordingly a variation in expressed brightness difference relative to a fixed range of tone values. Such non-linearity often appears in liquid crystal display devices, especially a liquid crystal display device that adopts a tone display technique called the frame skipping method.
The processing that compensates for the non-linearity means the general processing to relieve the deterioration of the picture quality due to the non-linearity. The processing includes a process of correcting the tone value to ensure the linearity between the tone value and the brightness and a process of correcting the tone value to prevent the occurrence of a quasi-contour due to the non-linearity. The quasi-contour is visually conspicuous, especially in a tone area having a wide brightness range relative to the width of the tone values, that is, in a tone area where the expressed brightness significantly varies with an increase of the tone value by xe2x80x981xe2x80x99. The latter tone correction is implemented, for example, by reducing the distribution of the number of pixels included in such a tone area, that is, by converting the tone values included in such a tone area into a tone value included in another tone area having a relatively narrow brightness range. A tone curve representing the mapping of the tone values before and after the processing may be used for the tone correction.
The above description regards the image processing according to the display characteristics of the image display device. In the case where a printing device is used as the image receiver device, the image processing may be carried out according to the printing characteristics of the printing device.
In one preferable application of the present invention, the image transmitter device receives processing characteristic information regarding image processing characteristics of the image receiver device from the image receiver device, and changes over the series of image processing executed therein according to the processing characteristic information. The processing characteristic information includes, for example, the type, the processing speed, and the memory capacity of the image receiver device.
In the case where an image output device is used for the image receiver device, the image transmitter device receives output characteristic information regarding image output characteristics of the image output device from the image output device and changes over the series of image processing executed therein according to the output characteristic information. The output characteristic information includes, for example, the type, the resolution, and the adopted color system of the image receiver device. Especially when an image display device is used for the image receiver device, the output characteristic information may include the adjusted contrast value and the environmental brightness. The environmental brightness may be measured with a sensor attached to the image receiver device or may be input manually by the user. In the case of a liquid crystal display device, the output characteristic information may include the on-off state of the backlight.
The changeover of the details of the image processing according to such information ensures improvement in picture quality with regard to diverse image receiver devices and under diverse working environments. The changeover may be attained by changing the series of image processing to be executed or by changing the parameters for specifying the details of the image processing. The former application selectively carries out both or either one of the tone correction process and the color reduction process according to the processing characteristic information or the output characteristic information. The latter application changes over the tone curve used for the tone correction according to the processing characteristic information or the output characteristic information.
The changeover of the details of the image processing is readily implemented by mapping the processing characteristic information or the output characteristic information to the details of the image processing to be executed or the parameters and storing the mapping. The information transmitted from the image receiver information may include the parameters used for the image processing. For example, a tone curve used for the tone correction may be transmitted from the image receiver device. This arrangement advantageously ensures the flexible specification of the image processing that is adequately executed by the image receiver device.
The present invention is also directed to the image receiver device, that is in combination with the image transmitter device discussed above.
The image receiver device of the present invention includes a receiver unit, an image processing unit, and a process controller. The receiver unit receives a series of data including image data via a communication path, which may be via wire or wireless. The communication path may be a wide-area network like the Internet, a LAN, or a local network between the image transmitter device and the image receiver device.
The image processing unit causes the input image data to be subjected to a specified series of image processing. The image processing includes a series of processing required for the image output as well as another series of processing to enhance the picture quality according to the characteristics of the image receiver device. It is not required that the image processing executed by the image processing unit of the image receiver device is equivalent to the image processing executed by the image transmitter device described above. The image processing executed by the image processing unit of the image receiver device is specified in an allowable range by the processing capacity of the image receiver device.
The process controller controls the series of image processing to be executed by the image processing unit, if any, in response to a preset control signal included in the series of received data. The control signal is generated by the image transmitter device. The process controller determines whether or not the input image data has previously undergone the image processing and identifies the details of the image processing previously executed, if any. In the case where no image processing has been carried out, the process controller flexibly applies a diversity of image processing executable by the image processing unit, in order to enhance the picture quality. In the case where some image processing has been carried out in advance, the process controller controls the image processing to be executed by the image processing unit, in order to prevent undesirable deterioration of the picture quality due to the interaction with the previously executed image processing.
In the latter case, one applicable procedure identifies the details of the image processing previously executed for the input image data in response to the control signal, and prohibits the image processing unit from executing an equivalent series of image processing. This arrangement effectively prevents deterioration of the picture quality, due to the duplicated execution of the identical image processing by the image transmitter device and the image receiver device.
In one application, the image receiver device with an image display unit having a smaller number of expressible tones than a number of tones included in the master image data carries out the dispersed-type color reduction process. In this application, it is preferable that the process controller prohibits the image processing unit from executing color reduction, when receiving the control signal that shows the image data has undergone the color reduction process. This arrangement desirably prevents the same color reduction process from being carried out in a duplicated manner.
In another application, the image receiver device with an image display unit having a non-linearity of expressible tones carries out the tone correction process. In this application, it is preferable that the process controller prohibits the image processing unit from executing an equivalent color correction process, when receiving the control signal that shows the image data has undergone the color correction process. This arrangement desirably prevents the same color correction process from being carried out in a duplicated manner.
The technique of the present invention is also applicable to a system where the image transmitter device is connected to a plurality of image receiver devices. In this application, the image transmitter device receives a specification of each image receiver device, changes over the details of image processing executed therein according to the input specification, and transmits processed image data to the corresponding image receiver device. A control signal for controlling details of image processing executed in each image receiver device is mapped to the processed image data to be transmitted. The specification of the image receiver device may include at least part of the processing characteristic information and the output characteristic information discussed previously. This arrangement ensures the image output according to the specification of each image receiver device under the environment where the plurality of image receiver devices are connected to the image transmitter device.
The technique of the present invention is not restricted to the image transmitter device or the image receiver device, but may be attained by a diversity of other applications. Such applications include a method of transmitting image data via the communication path to the image receiver device having the image processing functions and a method of generating image data to be transmitted. The applications also include a method of outputting images in the image receiver device, a method of controlling the image receiver device, and an image processing method in the image receiver device, as well as a pre-processing method of image data prior to transmission. The present invention may also be constructed as computer programs that attain these functions, a diversity of signals that are equivalent to such computer programs, and recording media in which such computer programs are recorded in a computer readable manner.
Typical examples of the recording media include flexible disks, CD-ROMs, magneto-optic discs, IC cards, ROM cartridges, punched cards, prints with barcodes or other codes printed thereon, internal storage devices (memories like a RAM and a ROM) and external storage devices of the computer, and a variety of other computer readable media.
These and other objects, features, aspects, and advantages of the present invention will become more apparent from the following detailed description of a preferred embodiment with the accompanying drawings.